Is Your Toxicity My Toxicity? Exploring the Impact of Rater Identity on Toxicity Annotation

Machine learning models are commonly used to detect toxicity in online conversations. These trained on datasets annotated by human raters. We explore how raters' self-described identities impact they annotate comments. first define the concept of Specialized Rater Pools: rater pools formed based identities, rather than at random. three such for this study - specialized raters from U.S. who identify as African American, LGBTQ, and those neither. Each these same set comments, which contains many references identity groups. found that is a statistically significant factor will identity-related annotations. Using preliminary content analysis, we examined comments with most disagreement between nuanced differences Next, annotations each different pools, compared scores several test sets. Finally, discuss using self-identify subjects can create more inclusive machine models, provide ratings random